Empirical Evaluation of Automatically Extracted Road Axes

Internal self-diagnosis and external evaluation of the obtained results are of major importance for the relevance of any automatic system for practical applications. Obviously, this statement is also true for automatic image analysis in photogrammetry and remote sensing. However, so far only relatively little work has been carried out in this area. This is mostly due to the moderate results achieved. Only recently automatic systems reached a state in which a systematic evaluation of the results seems to be meaningful. This paper deals with the external evaluation of automatic road extraction algorithms by comparison to manually plotted linear road axes used as reference data. The comparison is performed in two steps: (1) Matching of the extracted primitives to the reference network; (2) Calculation of quality measures. Each step depends on the other: the less tolerant is matching, the less exhaustive the extraction is considered to be, but the more accurate it looks. Therefore, matching is an important part of the evaluation process. The quality measures proposed for the automatically extracted road data comprise completeness, correctness, quality, redundancy, planimetric RMS differences, and gap statistics. They aim at evaluating exhaustivity as well as assessing geometrical accuracy. The evaluation methodology is presented and discussed in detail. Results of a comparative evaluation of three different automatic road extraction approaches are presented. They show the overall status of the road extractors, as well as the individual strengths and weaknesses of each individual approach. Thus, the applicability of the evaluation method is proven.